Guided Sequence-to-sequence Learning with External Rule Memory
نویسندگان
چکیده
External memory has been proven to be essential for the success of neural network-based systems on many tasks, including Question-Answering, classification, machine translation and reasoning. In all those models the memory is used to store instance representations of multiple levels, analogous to “data” in the Von Neumann architecture of a computer, while the “instructions” are stored in the weights. In this paper, we however propose to use the memory for storing part of the instructions, and more specifically, the transformation rules in sequence-tosequence learning tasks, in an external memory attached to a neural system. This memory can be accessed both by the neural network and by the human experts, hence serving as an interface for a novel learning paradigm where not only the instances but also the rule can be taught to the neural network. Our empirical study on a synthetic but challenging dataset verifies that our model is effective.
منابع مشابه
The effect of exposed to radiofrequencry emitted from jammer on the spatial memory
Introduction: The Widespread range of mobile phone application in life has induced many problems in the societies. This subject has caused jammer router is used in some places. Jammer router blocks signal transfer from base station to the cell phone. In the other hand, the extensive application of electromagnetic field in all aspect of life has increased fears and worries about...
متن کاملThe role of sleep on children's motor memory consolidation in a motor sequence task
Sleep has been shown to affect the consolidation of motor memory in adults. However, sleep-dependent consolidation is not well-specified in children as like as adults. So, the purpose of this study is to investigate the effect of sleep on childrenchr('39')s motor memory consolidation. The participants were 32 voluntary boys aging from 10 to 12 years old from city of Kerman, which had the necess...
متن کاملA two-stage stochastic rule-based model to determine pre-assembly buffer content
This study considers instant decision-making needs of the automobile manufactures for resequencing vehicles before final assembly (FA). We propose a rule-based two-stage stochastic model to determine the number of spare vehicles that should be kept in the pre-assembly buffer to restore the altered sequence due to paint defects and upstream department constraints. First stage of the model decide...
متن کاملContinuous Online Sequence Learning with an Unsupervised Neural Network Model
The ability to recognize and predict temporal sequences of sensory inputs is vital for survival in natural environments. Based on many known properties of cortical neurons, hierarchical temporal memory (HTM) sequence memory recently has been proposed as a theoretical framework for sequence learning in the cortex. In this letter, we analyze properties of HTM sequence memory and apply it to seque...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016